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Abstract — This paper introduces a new trellis pruning method 
which uses nonlinear convolutional coding for peak-to-average 
power ratio (PAPR) reduction of filtered QPSK and 16-QAM 
modulations. The Nyquist filter is viewed as a convolutional 
encoder that controls the analog waveforms of the filter output 
directly. Pruning some edges of the encoder trellis can effectively 
reduce the PAPR. The only tradeoff is a slightly lower channel 
capacity and increased complexity. The paper presents simulation 
results of the pruning action and the resulting PAPR, and 
also discusses the decoding algorithm and the capacity of the 
filtered and pruned QPSK and 16-QAM modulations on the 
AWGN channel. Simulation results show that the pruning method 
reduces the PAPR significantly without much damage to capacity. 

I. Introduction 

This paper presents a new coding-based method for peak- 
to-average power ratio (PAPR) reduction of Nyquist filtered 
M-ary Phase-Shift Keying (PSK) and M-ary Quadrature- 
Amplitude Modulation (QAM). Let x{t) denote the complex 
baseband signal after filtering and D/A conversion in the 
transmitter The PAPR is defined as 

max |a;(t)p 



V 



(1) 



where is the expected value of \x(t)\'^ [1]. 

In literature, PAPR reduction is mostly addressed and con- 
sidered as an important issue for OFDM systems. However, it 
is a substantial practical problem for Nyquist filtered mod- 
ulations as well. In a communication system link, due to 
bandwidth limitation, a pulse shaping filter is required after 
baseband modulation to suppress the sidelobes of the power 
spectra of PSK and QAM signals. Root Raised Cosine (RRC) 
filters are commonly used as the pulse shaping filter at the 
transmitter And the receiver uses an identical filter as matched 
filter so that the overall pulse shaping spectrum will be the 
regular Raised Cosine (RC) spectrum [2], which satisfies the 
Nyquist criterion for distortionless baseband transmission in 
the absence of noise [3]. The pulse shaping filter introduces 
envelope variation to PSK and amplifies envelope fluctuation 
of QAM. In the presence of envelope fluctuation, nonlinearity 
of the transmit high-power amplifier (HPA) will cause in- 
termodulation distortion and the sidelobes which have been 
removed by filtering will regrow. To prevent spectrum spread- 
ing, an HPA has to operate in the linear zone, which leads to 



low power efficiency [4]. Nonconstant envelope signals with 
pulse-shaping filters (for example PSK and QAM) have better 
spectrum efficiency than constant envelope signals. Yet people 
tend to use constant envelope modulations such as OMSK [5], 
CPM [6] and XPSK (also known as FQPSK) [7] to satisfy the 
power efficiency requirements. 

A coding-based method can effectively reduce the PAPR of 
filtered PSK and QAM signals and thus increases power effi- 
ciency. There are many coding-based PAPR reduction methods 
for OFDM in the literature, such as [8], [9], and [10]. However, 
these methods restrict the application of coding to the digital 
domain. This paper presents a new method employing a 
nonlinear convolutional coding strategy to control the analog 
waveform directly to achieve PAPR reduction. The RRC filter 
is viewed as a convolutional encoder Pruning certain edges 
of the encoder trellis can eliminate output waveforms that 
contribute to high PAPR. This trellis pruning method can be 
generally applied to all PSK and QAM modulations. QPSK 
and 16-QAM are considered as examples. Simulation results 
show that the pruning method yields especially good results 
when applied to 16-QAM, reducing the PAPR with negligible 
damage to capacity. Section |lll describes this new method, and 
some PAPR simulation results are presented in Section Hill 
Section Hvl discusses the decoding algorithm and the capacity 
of the new restricted channel. Section [V] concludes the paper 

II. Trellis Pruning Method 

Consider the root raised-cosine (RRC) pulse shaped QPSK 
modulation. The RRC shaping pulse is 



P{t) = 



(1 - (8am)2) 



sin(27rM^(l - a)t) 
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where a is the roll off factor and W is the ideal Nyquist 
bandwidth [2]. This paper considers a causal pulse shaping 
filter with finite length given by 



m 



\p{t-DT), ifO<t<2DT 
I 0, otherwise 



(3) 



where T is the symbol duration and the delay is D symbols. 
Truncating length of the filter is equal to twice the delay. The 
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Fig. 1. Trellis of an encoder with 1024 states 



Fig. 3. Pruned trellis (Pruned edges are shown in dashed lines.) 




-0.6 -0.5 -0.4 -0.3 -0.2 



-0.6 -0.5 -0.4 -0.3 -0.2 



-0.6 -0.5 -0.4 -0.3 -0.2 



-0.6 -0.5 -0.4 -0.3 -0.2 



Fig. 2. Output waveforms corresponding to the trellis shown in Fig.m 



filter output waveform is 



(4) 



i=0 



where {gi} is the input sequence with symbols drawn from 
QPSK constellation. 

We view the pulse shaping filter as a 4-ary convolutional 
encoder with 2D — 1 shift registers. Therefore, total number 
of states of the convolutional encoder is equal to 4^^^^. The 
output associated with the state transition initiated by the jth 



input QPSK message symbol gj is 

Sj ~ s{t) 

OG 
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i=j-(2D-l 



9iP{t - iT), 



(5) 
(6) 

(7) 



for jT < t < {j + 1)T. From (0, Sj is solely determined 
by the jth input and 2D — 1 message symbols stored in the 
shift register An example of the trellis of such an encoder 
with 1024 states (D=3) is shown in Fig. [2 The corresponding 
output waveforms are listed in Fig. |2] 

In 0, when giplt — iT) adds constructively, the trajectory 
of Sj goes far away from the origin; when gi'p{t — iT) 
adds destructively, the trajectory of Sj goes close to the 
origin. Therefore, some edges of the trellis produce zero- 
crossing output waveforms (as is shown in Fig. |2] (a)-(d)), 
and some produce peaks (Fig. |2(e)-(h)). Pruning such edges, 
i.e., eliminating the state transitions that generate large peak or 
zero-crossing, reduces the PAPR significantly. Take the trellis 
in Fig. n as example. Edge (d) and (h) will be pruned. The 
state transitions that follow the pruned edges will take one 
of the three remaining edges from the same state. The new 
trellis, which characterizes the pruning convolutional encoder, 
is shown in Fig. |3] 

The trellis pruning method for QPSK modulation also ap- 
plies to RRC filtered 16-QAM modulation. The only difference 
is that the pulse shaping filter is now viewed as a 16-ary 
convolutional encoder. 



III. PAPR Results 

This section provides PAPR reduction results using the 
method described in Section [II] There are many pruning 
strategies to choose from. This paper uses the following 
strategy; edges of the trellis that produce output waveforms 
going farthest away from the origin (peak) and passing nearest 
to the origin (zero-crossing) are simultaneously pruned. We 
use pruning percentage ?/ to measure how much pruning is 
actually performed. 77 is defined as 



OPSK PAPR vs. Pruning Percentage 



v = 



number of pruned edges 
total number of edges in the trellis 



(8) 



All the filters used in the simulations have a delay D of 3 
symbols. Fig.0]shows the trajectory diagrams of RRC filtered 
QPSK signals without pruning and with rj 10%, 30%, 50% 
pruning respectively. It is clear that with the increase of the 
pruning percentage, zero-crossing and peak waveforms are 
eliminated. 
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Fig. 5. PAPR vs. pmning percentage for RRC filtered QPSK 



16-QAM PAPR vs. Pruning Percentage 
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Fig. 6. PAPR vs. pruning percentage for RRC filtered 16-QAM 



Fig. 4. Trajectory diagrams of RRC filtered QPSK (a)w/o pruning, (b)10% 
pruning, (c)30% pruning, and (d)50% pruning. 

The above pruning strategy is simulated for both QPSK and 
16-QAM for various pruning percentage, where the shaping 
filters are the RRC filters of different roll-off factors. Fig.|5]and 
Fig. |6l shows that the pruning method can effectively reduce 
the PAPR of both modulation schemes. The reduction of the 
PAPR is more significant for 16-QAM: a 0.5% pruning can 
reduce the PAPR by more than 25%. 

IV. Decoding Algorithm and Capacity 

A. Decoding Algorithm 

Viterbi algorithm is typically used to decode convolutional 
codes. However, the number of states of the encoder presented 
in this paper grows exponentially with the length of the pulse 
shaping filter and constellation size. With the increase of 
filter length and the expansion of constellation from QPSK 
to 16-QAM, the complexity of Viterbi algorithm makes it 
computationally intractable. Therefore, we use a forward only 



decision feedback aided BCJR algorithm with much lower 
complexity as the decoding algorithm. 

We model the pruning convolutional encoder as a finite- 
state machine (FSM) [11]. The concatenation of the FSM 
and the Additive White Gaussian Noise (AWGN) channel, 
as is shown in Fig. is modelled as a Finite State Markov 
Channel (FSMC). We call this particular FSMC the pruning 
channel. The channel input process {Xk} is assumed to be 
i.i.d. symbols drawn from a finite set S. The state process 
{Sk} forms an irreducible, aperiodic, stationary finite state 
Markov chain over a finite state space Q = {1, • • • 1 A'}. The 
statistics of the FSMC satisfies the following properties; 

. P{Xk\S^) = P{Xk) for I < k; 

m Given X^ and Sk, {Yk, Sk+i} is statistically independent 
of {l^.}ti\and {S,}tl 

For FSMC, the standard BJCR [12] calculates the a poste- 
riori probability (APP) of Xk given past and future channel 
outputs {Yi}^^. Whereas, the decision feedback aided BCJR 
(DFA-BCJR), proposed in [13] and [14], calculates the APP 
value of Xk given a finite sequence of the past channel inputs 
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Fig. 7. The pruning channel model 

{Xiy'l'l and a finite sequence of past and future channel 
outputs {rj^i. Denote = and Y^^ = {Fj^i 

for notational convenience. From the Bayes rule, 



P{Yf\X^)P{Xk) 



(9) 



We can rewrite the conditional probabilities as 

P{Y,^\X^)^Y.J2''k{s'hk+i{s',s,Xk)Pk+i{s), (10) 

s' s 

P(yi^lX^-i) = ^^afc(s')7fc+i(.s',5)/3fc+i(s), (11) 



Where 

akis')=PiY,'-\Sk = s'\Xt') 



(12) 



^k+iis\s,Xk)^P{Yk,Sk+i^s\Sk^s\Xk), (13) 
jk+i{s',s) = P{Yk,Sk+i = s\Sk = s'), (14) 
(3k+i{s)^P{Y,l,\Sk+i=s). (15) 

afc denotes the forward path of the DFA-BCJR algorithm, 
while /3fc denotes the backward path. DFA-BCJR algorithm 
differs from the standard BCJR algorithm due to the presence 
of known input symbols in the forward path. Note that when 
given X'^^^, il2\ can be rewritten as 



Q!fc(s') 



1 , if s' takes on value determined by X^ 
0, otherwise 



fc-i 



(16) 



Substitute ([T6j into and iU\ . 

= ^7fc+i(s',s,Xfc)/3fc+i(s), (17) 



(18) 



where the state s' takes on value determined by X^^^. 

Equation ( I17> and (I18> show that with decision feedback, 
the complexity of the forward path calculation is independent 
of the number of states K of the FSMC. However, the 
complexity of the backward path still grows exponentially with 
K. To further reduce complexity, we only use the forward path 
of the DFA-BCJR as the decoding algorithm. That is, when 
calculating the APP value of Xk, no future channel outputs 
{Yk}iLk+i available. Therefore, the APP value becomes 

From it is clear that the decoding algorithm has a 

complexity growing linearly with the number of states. 



To achieve good performance of the decoding algorithm, we 
need to carefully choose the appropriate coding scheme before 
the convolutional encoder, such that accurate estimation of the 
previous input symbols can be obtained and then feedback to 
the decoding algorithm as known input symbols. 

B. Capacity and Capacity Lower Bound 

This section discusses how to calculate the capacity and 
capacity lower bound of the pruning channel. Simulation 
results show that pruning does not decrease capacity much. 
Here capacity refers to the constrained channel capacity of a 
given input distribution. We only consider M -ary i.i.d. inputs. 

The capacity of the pruning channel, which is modelled as 
an FSMC shown in Fig. is 



C 



1 



(20) 



Expanding j^I{X^ -jY-^), we have 

= ^[H{X^) H{X^\Y,^) 



log2 M 



1 



N 



-Y^H{X,\Y,^,Xl~^] 

k=l 
N 



(21) 



log2 M-^Y. l°g2 P{Xk\Y,'', Xf)] ■ (22) 



fe=i 

As N goes to infinity, i22\ is the capacity of the pruning 
channel. P{Xk\Yf , X^~^) in (|23 can be computed by the 
DFA-BCJR algorithm. Therefore, we can calculate the channel 
capacity by Monte Carlo simulation of equation (|9jl to dlSI l and 
(|23. 

The computation becomes too complex when the filter 
length or M is large. To simplify calculation, we derive a 
capacity lower bound as follows. Since conditioning reduces 
entropy, we have 



H{Xk\Yf,Xt') < H{Xk\Y^,Xl 
Substituting (|23} into (|2T) yields 



1 

TV 



I{X^;YI') 



1 ^ 

>iog2Af--^i/(Xfe|r^xf-i^ 

fe=i 



lo: 



1 ^ 



(23) 



(24) 



(25) 



(26) 



fc=i 



Equation (I26t is the capacity lower bound of the pruning 
channel. It can be simulated using equation M9\ . This lower 
bound is also the achievable rate of the forward only DFA- 
BCJR decoder discussed in Section IIV-AI 

Capacity of the pruning channel with QPSK inputs and 
capacity lower bound of the pruning channel with 16-QAM 
inputs are shown in Fig.|8]and Fig.|9]respectively. These results 
illustrate that, capacity loss, compared with the normal AWGN 
channel without pruning, is fairly small when pruning is less 



Fig. 8. Capacity of the AWGN channel and the 10%, 30%, 50% pruning 
channel with QPSK inputs 



Fig. 10. Percentage of capacity remaining after pruning vs. PAPR for RRC 
filtered 16-QAM signals 
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Fig. 9. Capacity of the AWGN channel and capacity lower bound of the 
1%, 4%, 7% and 10% pruning channel with 16-QAM inputs 

than 10%. For 16-QAM, 10% pruning reduces the PAPR by 

more than a half. Define 

Capacity of the pruning channel 

P = (27) 

Capacity of the normal AWGN channel 

to represent the amount of capacity that is preserved after 
pruning. Fig. which combines Fig. |S] and Fig. |5] further 
confirms that with a small loss of capacity, the pruning method 
can achieve significant PAPR reduction for filtered 16-QAM 
modulation. 

V. Conclusion 

This paper introduces a new trellis pruning method for 
reducing the PAPR of filtered QPSK and 16-QAM signals. 
The pulse shaping filter is viewed as a nonlinear convolutional 
encoder. Eliminating certain state transitions of the encoder 
can reduce the PAPR of transmitted signals. Simulation results 
confirm that this method can significantly reduce the PAPR of 
both QPSK and 16-QAM modulations. 

Forward only DFA-BCJR algorithm is used as the decoding 
algorithm. DFA-BCJR algorithm also helps to calculate the 



capacity of the pruning channel. Capacity loss is small when 
pruning is less than 10%, which means for 16-QAM, reducing 
the PAPR by more than a half only costs a negligible capacity 
loss. 
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